## Replication repository

*"Priming bias versus post-treatment bias in experimental designs" by Matthew Blackwell, Jacob R. Brown, Sophie Hill, Kosuke Imai, and Teppei Yamamoto*

 

#### Notes

This replication repository generates the analysis and figures used in the paper and supplemental materials. The script `run_all.R` runs all scripts in the correct order. It will also create directories for output and figures, if they are not already created. (Note that it takes approximately 6 hours to run all the scripts.)

The raw data is from Horowitz & Klaus (2018), which is available on the [Harvard Dataverse](https://doi.org/10.7910/DVN/XBWR8N).

Note: These scripts call functions from the [prepost](https://github.com/mattblackwell/prepost) package, which implement the non-parametric bounds and Bayesian methods.

`FiguresandTables.xlsx` contains information on each figure and table in the manuscript.

 

#### Overview

-   `hk-cleaning.R` cleans the data

-   `hk-analysis.R` and `hk-figures.R` generate Figures 1-5 in the paper

-   `hk-prior-predictive.R` generates Figure SM.1

-   `hk-replication-priors.R` generates Figure SM.2

-   `hk-replication-plot2.R` generates Figure SM.5

-   `hk-replication-plot3.R` generates Figure SM.6

-   `gibbs_functions.R` and `gibbs_sim_data.R` contain functions for the Gibbs simulations

-   `gibbs-assumptions-simulation.R`, `gibbs-covariate-simulation-betas.R`, `gibbs-covariate-simulation-psis.R` run the Gibbs simulations

-   `sim-visualizations.R` loads the simulation results and generates Figures SM.3 and SM.4

 

#### Script inputs and outputs

+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| Script                             | Description                                                                        | Input                        | Output                                              |
+:===================================+:===================================================================================+:=============================+:====================================================+
| hk-cleaning.R                      | Cleans Horowitz-Klaus replication data                                             | data/Horowitz-Klaus-data.dta | output/hk-data-cleaned.RData                        |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| hk-analysis.R                      | Runs OLS, nonparametric bounds, and Gibbs sampler, as well as sensitivity analyses | output/hk-data-cleaned.RData | output/hk-replication.Rdata                         |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| hk-figures.R                       | Generates Figures 1-5                                                              | output/hk-replication.Rdata  | figures/figures/hk-plot1.pdf (Fig 1)                |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | figures/hk-plot2-non-param.pdf (Fig 2)              |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | figures/hk-plot-sens.pdf (Fig 3)                    |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | figures/hk-plot-pre-sens.pdf (Fig 4)                |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | figures/hk-plot-prepost-sens.pdf (Fig 5)            |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| hk-prior-predictive.R              | Prior predictive distribution                                                      | output/hk-data-cleaned.RData | figures/ppd.pdf (Fig SM.1)                          |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| hk-replication-priors.R            | Gibbs sampler with different priors                                                | output/hk-data-cleaned.RData | figures/hk-priors-plot.pdf (Fig SM.2)               |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    | output/hk-replication.Rdata  |                                                     |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| hk-replication-plot2.R             | Comparing bounds and Bayesian results                                              | output/hk-replication.Rdata  | figures/hk-plot2.pdf (Fig SM.5)                     |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| hk-replication-plot3.R             | Effect of covariates on Bayesian results                                           | output/hk-replication.Rdata  | figures/hk-plot3.pdf (Fig SM.6)                     |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| gibbs_functions.R                  | Functions from Gibbs sampler                                                       |                              |                                                     |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| gibbs_sim_data.R                   | Function for simulated data                                                        |                              |                                                     |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| gibbs-assumptions-simulation.R     | Gibbs simulations (different assumptions)                                          |                              | output/sim-monotonicity-TRUE-stable-TRUE-output.csv |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| gibbs-covariate-simulation-betas.R | Gibbs simulations (effect of covariates)                                           |                              | output/sim-weak-betas-output.csv                    |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | output/sim-med-betas-output.csv                     |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | output/sim-strong-betas-output.csv                  |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| gibbs-covariate-simulation-psis.R  | Gibbs simulations (effect of covariates)                                           |                              | output/sim-weak-psis-output.csv                     |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | output/sim-med-psis-output.csv                      |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | output/sim-strong-psis-output.csv                   |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+
| sim-visualizations.R               | Generates Figures SM.3 and SM.4                                                    | Simulations                  | figures/sim-plot-assumptions.pdf (Fig SM.3)         |
|                                    |                                                                                    |                              |                                                     |
|                                    |                                                                                    |                              | figures/sim-plot-covariates.pdf (Fig SM.4)          |
+------------------------------------+------------------------------------------------------------------------------------+------------------------------+-----------------------------------------------------+

#### References

Horowitz, Jeremy; Klaus, Kathleen, 2018, "Replication Data for: 'Can Politicians Exploit Ethnic Grievances? An Experimental Study of Land Appeals in Kenya'", <https://doi.org/10.7910/DVN/XBWR8N>, Harvard Dataverse, V1
